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ABSTRACT 


This  is  a proposal  to  develop  a software  test  service  at  the  National  Institute  of 
Standards  and  Technology  for  use  in  testing  the  accuracy,  or  numerical  precision, 
of  mathematical  software  for  special  functions.  The  service  would  use  the  World 
Wide  Web  to  receive  test  requests  and  return  test  results.  The  tests  would  be 
run  on  a network  of  workstations  at  the  Institute.  It  is  hoped  that  such  a service 
will  be  of  practical  utility  to  anyone  who  uses  special  functions  in  physics  or  other 
applications,  and  that  it  will  stimulate  the  interest  of  applied  mathematicians 
who  are  interested  in  the  computation  of  special  functions  as  well  as  computer 
scientists  who  are  interested  in  innovative  uses  of  the  Internet.  The  author 
solicits  comments  on  any  aspect  of  the  proposed  service. 
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1 Introduction 


Mathematical  software  is  deeply  embedded  in  the  computing  environment.  Since 
this  environment  is  evolving  rapidly,  its  impact  on  mathematical  software  needs 
to  be  revisited  regularly. 

Progress  in  parallel  computing  has  stimulated  much  reworking  of  numerical 
algorithms,  particularly  in  computational  linear  algebra.  The  earlier  introduc- 
tion of  vector  computers  had  a similar  effect.  Recent  advances  in  communica- 
tions and  networking  have  led  to  the  global  interconnection  of  computers  by 
high  bandwidth  communication  links,  one  result  of  which  is  improved  access 
to  information  about  mathematical  software  via  the  Internet  and  World  Wide 
Web.  For  example,  electronic  catalogs  and  repositories  such  as  xnetlib  [8]  are 
now  consulted  routinely  for  help  in  locating  and  obtaining  mathematical  soft- 
ware packages  for  specific  tasks. 

Vector  and  parallel  developments  have  had  only  a modest  impact  on  the  com- 
putation of  mathematical  functions.  Some  references  can  be  cited,  for  example 
[4]  and  [10].  However,  mathematical  functions  seem  particularly  appropriate  for 
demonstrating  a new  and  potentially  valuable  use  of  the  Web:  mathematical 
software  testing.  The  problem  of  testing  is  intrinsically  simpler  for  mathemat- 
ical functions  than  for  other  kinds  of  numerical  computation.  The  input  and 
output  Euclidean  spaces  have  low  dimension.  In  contrast,  numerical  linear  al- 
gebra deals  with  Euclidean  spaces  of  high  dimension,  and  most  other  numerical 
computations  deal  with  function  spaces  of  infinite  dimension.  Test  procedures 
for  mathematical  functions  can  be  devised  that  apply,  in  theory,  to  all  possible 
inputs. 

A question  that  needs  answering  at  this  point  is:  What  would  be  the  ad- 
vantages, compared  to  current  testing  practice,  in  using  the  Web  to  test  math- 
ematical software?  Our  answer  is  that  tests  can  be  tailored  to  suit  a particular 
need,  and  they  can  be  performed  on  demand. 

To  see  why  this  is  useful,  we  divide  current  practice  into  two  categories. 
Supplier  testing  is  performed  by  the  software  writer  or  project  team.  Referees, 
editors  and  software  maintainers  also  play  a role.  These  people  have  direct  re- 
sponsibility for  the  correctness  of  the  software.  For  commercial  software,  high 
license  fees  are  justified  largely  by  the  high  costs  associated  with  software  main- 
tenance and  testing.  Independent  testing  is  performed  by  other  individuals  and 
groups.  Users  often  perform  this  kind  of  testing  for  their  own  purposes  because 
it  leads  to  an  increased  confidence  in  the  correctness  of  the  software.  Some- 
times independent  tests  are  conducted  and  published  in  journal  articles  and 
institutional  reports  as  a guide  for  prospective  users. 

Published  tests  of  either  kind  have  a ‘frozen-in-time’  quality  about  them, 
having  been  performed  at  some  time  in  the  past  in  a computing  environment 
that  may  be  very  different  from  the  prevailing  one.  Even  more  unsettling,  since 
tests  are  never  complete,  their  results  may  not  apply  directly  to  the  numerical 
computation  of  current  interest. 


1 


Just  the  simple  repetition  of  a mathematical  function  test  is  often  imprac- 
tical because  the  test  program  is  unavailable.  Notable  exceptions  are  the  test 
programs  of  Cody  and  Waite  [7]  and  Cody  [5]  for  elementary  functions  of  real 
and  complex  arguments,  and  of  Cody  [6]  for  special  functions  of  real  argument. 
Here  we  are  following  the  conventional  terminology  of  calling  the  transcendental 
functions  met  in  calculus  courses  elementary  functions  and  the  higher  functions 
that  appear  in  advanced  applications  special  functions.  The  general  unavail- 
ability of  test  programs  is  undoubtedly  related  to  the  considerable  effort  that  is 
required  to  raise  them  to  an  acceptably  high  standard  for  publication  or  public 
distribution.  Another  problem  is  lack  of  generality.  For  example,  most  test 
programs  apply  only  to  a builtin  set  of  test  arguments,  often  with  an  element 
of  randomness  included. 

In  this  paper  a software  test  service  for  special  functions  is  proposed  and 
some  implementational  details  are  given.  The  emphasis  is  on  special  functions, 
because  this  is  where  the  need  is  greatest,  but  the  service  will  apply  equally  well 
to  elementary  functions.  The  service  will  provide  a tool  that  can  be  used  to 
tailor  tests  to  specific  requirements.  Therefore  it  should  be  of  interest  in  both 
supplier  and  independent  testing. 

2 Proposal 

The  purpose  of  the  proposed  software  testing  service  for  special  functions  is  to 
assess  the  accuracy,  or  numerical  precision,  of  computed  function  values  through 
the  use  of  a comparison  method.  Test  values  will  be  compared  against  reference 
values  computed  in  higher  precision  by  highly  accurate  algorithms.  Test  re- 
quests will  be  submitted  to  a Web  server  at  the  National  Institute  of  Standards 
and  Technology.  The  tests  will  be  conducted  at  the  Institute  using  software 
developed  for  the  purpose.  The  test  results  will  be  returned  to  the  requester  in 
the  form  of  an  appropriate  document  on  the  Web  server. 

The  key  components  of  the  service  will  be 

Reference  Software  This  will  consist  of  highly  accurate  and  reliable,  but  not 
necessarily  efficient,  numerical  procedures  for  generating  high-precision 
reference  values  of  special  functions  over  very  extensive  argument  domains. 
The  reference  software  will  be  an  excellent  repository  for  advanced  algo- 
rithms because  it  will  be  embedded  in  a computing  environment  that  mit- 
igates the  computer  arithmetic  liabilities  (underflow,  overflow,  and  limited 
precision)  of  conventional  computing  environments. 

Comparison  Software  This  will  serve  the  purpose  of  orchestrating  the  gener- 
ation of  reference  values  and  determining  the  precision  of  test  values.  The 
comparison  software  will  utilize  parallel  methods  via  the  simple  device  of 
domain  partitioning.  An  appropriate  measure  of  precision  will  be  defined 
in  terms  of  interval  mathematics. 
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Communication  Interface  This  will  be  an  appropriately  designed  Web  doc- 
ument with  associated  subdocuments  for  accepting  test  requests  and  re- 
turning test  results  via  the  Internet. 

The  comparison  method  was  chosen  because  of  its  conceptual  simplicity.  An 
alternative  approach,  numerical  verification  of  identities,  is  advocated  and  used 
by  W.  J.  Cody  and  his  coworkers.  It  avoids  the  need  for  higher  precision  but  it 
requires  careful  attention  in  the  choice  of  identity  to  guarantee  against  incorrect 
conclusions  that  could  arise  if  the  identity  were  not  entirely  independent  of  the 
algorithm  used  in  the  implementation  of  the  function.  Also,  care  must  be  taken 
to  separate  the  error  in  the  function  evaluation  from  the  error  in  the  evaluation 
of  the  identity.  These  complications  will  be  avoided  in  the  test  service  by  taking 
full  advantage  of  the  tremendous  power  of  current  capabilities  for  computation 
and  communication. 


3 Reference  Software 

The  reference  software  is  at  the  heart  of  the  proposed  software  testing  system 
for  special  functions.  Not  only  must  it  be  highly  accurate,  a definite  bound  on 
the  error  in  each  computed  reference  value  is  essential.  Otherwise,  no  one  can 
be  certain  of  the  results  of  a test.  For  this  reason,  the  reference  software  should 
be  written  using  interval  techniques.  An  introduction  to  interval  computations 
is  given  in  the  book  by  Alefeld  and  Herzberger  [2].  However,  aside  from  the 
elementary  functions,  very  little  has  been  published  on  interval  algorithms  for 
specific  mathematical  functions.  An  opportunity  and  a need  exists  here  for 
numerical  analysts  to  develop  interval  algorithms  that  generate  the  required 
error  bounds. 

The  service  must  be  able  to  test  double-precision  as  well  as  single-precision 
software.  Thus  it  is  appropriate  to  write  the  reference  software  in  multiple 
precision.  The  Fortran  package  of  Bailey  [3]  is  available  and  applicable  for  this 
purpose.  Because  of  its  vast  exponent  range  in  comparison  to  conventional  com- 
puter arithmetic  systems,  Bailey’s  package  relieves  the  need  to  be  careful  about 
underflow  and  overflow.  The  occurrence  of  these  conditions  can  completely  in- 
validate an  otherwise  pristine  computation.  The  algorithms  will  take  fully  into 
consideration  stability  and  roundoff  questions  because  these  too,  if  ignored,  can 
destroy  a computation. 

Highly  efficient  software,  at  least  for  functions  of  one  variable,  is  usually 
precision-limited  because  it  employs  polynomial  or  rational  approximations  that 
are  constructed  with  respect  to  a fixed  target  precision.  Flexibility  is  more  im- 
portant than  efficiency  for  reference  software.  Ideally,  reference  algorithms  will 
accept  an  arbitrary  tolerance  specification  so  that  the  same  programs  can  be 
executed  in  increased  precision  without  a major  effort  to  generate  approxima- 
tion coefficients  for  the  higher  precision.  This  means  that  methods  will  be 
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constructed  from  Taylor  expansions,  asymptotic  expansions,  differential  or  dif- 
ference equations,  integral  representations,  and  other  analytical  properties,  just 
as  is  done  in  much  existing  software  for  functions  of  two  or  more  variables. 


4 Comparison  Software 

The  comparison  software  hcis  a mathematical  component  and  a computer  sci- 
ence component.  The  mathematical  component  is  concerned  with  measuring 
the  error  in  test  values.  This  could  be  done  simply  with  pointwise  absolute  or 
relative  error  but  an  interval  formulation  is  more  appropriate.  The  computer 
science  component  is  needed  to  collect  and  process  the  test  and  reference  val- 
ues. This  is  a natural  application  for  parallel  processing  with  a loosely  coupled 
network  of  computer  workstations. 

Only  the  mathematical  component  will  be  considered  here.  It  is  easy  to 
describe,  at  least  when  all  variables  are  real.  Let  us  consider  a function 

y = f{x),  yeTi,  (1) 

where  TZ  denotes  the  set  of  real  numbers.  Let  T be  the  set  of  real  numbers 
that  are  representable  exactly  in  the  format  of  a particular  computer  arithmetic 
system,  excluding  any  nonnumerical  symbolic  representations  such  as  ±oo,  ±0 
and  NaN  (Not-a-Number).  Thus  an  approximating  function 

y = f{x),  xeT^,  y^T  (2) 


is  defined  by  the  software  to  be  tested.  Our  problem  is  to  measure  the  error 
committed  when  y is  taken  as  an  approximation  to  y. 

The  pointwise  absolute  error,  defined  for  x E ■,  is  just  \y  — y\-  Because 
absolute  error  is  naturally  associated  with  fixed-point  computation,  and  not 
floating-point,  relative  error  is  more  appropriate  except  near  zeros  of  the  func- 
tion /.  Instead  of  relative  error  p — \{y—y)ly\i  we  prefer  to  use  relative  precision 


|in(y/y)l 

undefined 


if  yy  > 0, 
otherwise; 


(3) 


this  definition  was  introduced  in  [13].  Since  rp(y,  y)  = p 0{p^),  relative 
precision  and  relative  error  are  nearly  the  same  when  y is  a good  approximation 
to  y.  But  relative  precision  has  the  advantage  for  detailed  error  analyses  that  it 
is  a metric  on  TZ'^  and  TZ~ , where  these  symbols  denote  the  open  real  intervals 
(0,oo)  and  (— oo,0),  respectively. 

Given  x E ^ the  exact  function  value  y = /(x)  determines  the  interval 
^ — [Vtiyu]  where  yi,yu  are  two  consecutive  elements  of  T.  A criterion  that 
is  applied  sometimes  in  the  construction  of  computer  software  is  to  require  the 
approximate  function  value  y = S{x)  to  satisfy  either  y = yi  oi  y = y^.  This 
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will  be  called  the  criterion  of  full  precision.  It  can  be  expressed  in  a different 
way.  First  we  define  the  machine  epsilon 

e = maxrp(t,  <■*■).  (4) 


where  4+  denotes  the  successor  of  t in  T.  Then  the  approximating  function  / 
satisfies  the  criterion  of  full  precision  if,  and  only  if,  rp(y,  y)  < e for  all  x G 
such  that  f{x)  and  /(x)  have  the  same  sign.  It  is  customary  to  employ  full 
precision,  or  nearly  full  precision,  in  software  for  elementary  functions. 

The  strongest  possible  criterion  is  correct  rounding.  Required  by  [9]  for 
standard  floating-point  arithmetic  operations,  it  is  met  by  most  up-to-date  im- 
plementations of  floating-point  arithmetic  as  well  as  by  some  software  for  ele- 
mentary functions,  particularly  when  supplied  with  Fortran  compilers. 

The  criterion  of  full  precision  is  quite  rigorous.  If  x G let  x~  and  x"*" 
denote  its  predecessor  and  successor  (where  the  ordering  is  defined  component- 
wise). If  we  regard  x as  a representative  of  the  multivariate  interval 


X = [x^,x^]  = -[x  + X ,x  + x+]. 


(5) 


then  the  uncertainty  in  x is  reflected  in  the  range  of  / as  its  arguments  vary 
throughout  X.  If  a partial  derivative  of  / is  large,  it  can  be  argued  that  it  is 
unnecessary  to  require  full  precision  in  f{x).  In  fact,  the  computation  of  f[x) 
to  full  precision  is  unwarranted  if  it  requires  an  inordinate  amount  of  execution 
time.  This  penalty  is  likely  to  be  especially  severe  for  special  functions. 

A more  appropriate  criterion  of  precision  can  be  defined.  Let  I {TV}  be  the 
set  of  all  closed  interval  subsets  of  TZ.  Alefeld  and  Herzberger  [2]  show  that,  if 
A = [at,au]  and  B — [bi,  are  two  intervals,  then  the  function 


g(A,  B)  = max{|a^  - 6^1,  |ou  - 6u|},  A,BeI{TZ)  (6) 

is  a metric.  Also,  since  g([a,  a],  [6,  6])  = |a  — &|,  the  metric  q generalizes  the  usual 
metric  in  TZ.  Arithmetic  operations  A + B,  A — B,  AB  and  A/B  are  defined  in 
I{TZ)  by  operating  on  the  endpoints  of  the  intervals.  They  are  continuous  in  the 
topology  of  {I{TZ),q}.  Similarly,  it  is  possible  to  define  continuous  interval  ex- 
tensions of  continuous  real  functions.  For  example,  for  the  logarithmic  function, 
the  interval  extension  ln(A)  = [lna/,lnau]  is  defined  and  continuous  on  I{TZ^). 
Next,  we  define  interval  relative  precision 

r g(lny,lny)  ify,y  G/(72+), 

rp(y,y)=  rp(-y,-y)  if  y,y  g /(7e-),  (7) 

[ undefined  otherwise. 

This  is  easy  to  compute,  since  it  can  be  shown  that 

rp(y,y)  = max{rp(y^,yi),rp(yj*,yu)}.  (8) 
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Interval  relative  precision  is  a metric  on  and  and  it  generalizes 

pointwise  relative  precision  since  rp([y,  y],  [y,  y])  = rp(y,  y). 

Now  consider  the  test  argument  x G again  as  a representative  of  the 

multivariate  interval  X = [x^,Xu],  and  assume  the  function  / is  continuous  on 
X.  Let  Y be  the  range  of  / on  X: 

y = [yi,  Vu]  = f{X)  = {/(x)  I < X < Xu}.  (9) 

Finally,  let  the  test  function  value  y = /(x)  G X represent  the  interval 

Y = [yi,yu]  = ^[y  + y~,y  + y'^].  (lo) 

Then  we  will  say  that  the  approximating  function  f satisfies  the  interval  crite- 
rion of  -precision  if 

rp(5^,  Y)  < max{e,  rp(y^,  y^)}  (H) 

for  all  X such  that  the  relative  precisions  are  defined.  The  right  side  of  this 
inequality  provides  a standard  of  comparison.  It  takes  into  account  the  behavior 
of  / as  its  arguments  vary  throughout  the  neighborhood  represented  by  x.  It 
establishes  the  allowable  range  of  relative  errors  over  this  neighborhood.  The 
left  side  measures  the  distance  between  the  allowable  range  of  / and  the  interval 
represented  by  the  test  function  value.  If  the  interval  criterion  (11)  is  satisfied, 
then  the  set  intersection  T n T is  nonempty.  If  Y"  QY  or  T C Y,  then  (11)  is 
satisfied.  In  all  cases  when  (11)  is  satisfied,  a simple  interpretation  in  terms  of 
pointwise  relative  error  can  be  given.  This  will  be  discussed  in  a future  paper. 

A fundamental  problem  in  interval  mathematics  is  how  to  compute  the  range 
of  real  functions.  Evaluation  of  explicit  expressions  using  interval  arithmetic 
does  not  necessarily  produce  the  range;  to  the  contrary,  the  range  may  be  over- 
estimated substantially.  Also,  it  is  necessary  to  construct  strict  bounds  for  all 
errors  caused  by  truncating  infinite  processes.  This  problem  will  need  to  be 
faced  in  the  design  and  construction  of  reference  software  for  the  software  test 
service. 


5 Communication  Interface 

For  the  software  test  system,  an  argument  set  is  a subset  of  the  domain  of  a 
function.  For  each  test,  the  test  requester  provides  an  argument  set  together 
with  corresponding  function  values  to  the  communication  interface.  Then  the 
reference  software  computes  the  function  to  higher  precision  at  all  points  in  the 
argument  set,  the  comparison  software  compares  the  reference  values  against  the 
test  requester’s  function  values,  and  finally  the  communication  interface  returns 
a table  or  plot  of  the  interval  relative  precision  to  the  test  requester. 

A careful  development  of  the  software  test  service  requires  attention  to  the 
processes  of  decimal-to-binary  and  binary-to-decimal  conversion.  Base  conver- 
sion processes  between  arbitrary  bases  are  considered  in  detail  in  Matula  [11] 
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and  [12].  A p-digit,  base-/3  significance  space  is  the  set  of  all  p-digit  normal- 
ized floating-point  numbers  in  the  base  j3,  excluding  zero  and  without  regard  to 
size.  Let  and  be  two  significance  spaces.  The  rounding  conversion  map- 
ping Rl  from  into  SI  is  the  mapping  that  is  defined  by  converting  x G S^ 
into  its  ly-ary  expansion  to  sufficiently  high  precision,  then  rounding  it  to  q 
base-i^  digits.  The  truncation  conversion  mapping  TJ  is  defined  similarly.  The 
composition  of  base  conversion  mappings  is  possible.  An  interesting  kind  of 
composition  is  an  in-and-out  conversion  mapping  which  maps  into  5^,  and 
then  back  to  S^.  Matula  proved  two  theorems: 

Theorem  1 (Base  Conversion  Theorem)  ///?*  ^ for  any  positive  inte- 
gers i,  j,  then  the  base  conversion  mappings  Rl  : —*  SI  and  : S^  SI 

are: 

1.  one-to-one  onto  their  ranges  if  and  only  if  ~ 1/ 

2.  onto  if  and  only  if  — 1. 

Theorem  2 (In-and-Out  Conversion  Theorem)  If  for  any  posi- 

tive integers  i,  j,  then 

1.  R^Rl  : 5^  — » is  the  identity  if  and  only  if  > /3^ , and 

2.  RF^T^  : S^  S^  is  the  identity  if  and  only  if  > 2/3^  — 1. 

The  condition  for  any  positive  integers  i,  j excludes  the  trivial  case 

when  the  bases  P and  v are  integral  powers  of  a common  base.  Under  the 
conditions  of  Theorem  2,  Rl  and  TJ  are  one-to-one  onto  their  ranges  and  their 
inverse  mappings  coincide  with  : S®  — > S^. 

As  an  example,  consider  decimal  output  from  single-precision  computer 
arithmetic  as  defined  in  [9].  Then  /3  = 2,  p = 24,  v — 10,  and  q is  to  be 
determined  according  to  some  criterion.  The  rounding  conversion  mapping 
S?o  is 

1.  one-to-one  onto  its  range  if  and  only  if  9 > 9; 

2.  onto  if  and  only  if  9 < 6; 

similarly  for  T^q.  Thus  decimal  output  precision  9 need  not  exceed  9 digits  if 
each  internal  number  is  to  have  a unique  decimal  representation,  and  it  cannot 
exceed  6 digits  if  the  complete  set  of  9-digit  decimal  numbers  is  to  be  covered. 
Also,  either  of  R^'^RIq  or  R^^T^q  is  the  identity  mapping  if  and  only  if  9 > 9. 

Now  let  denote  the  significance  space  associated  with  the  test  requester’s 
computer  arithmetic  T.  Let  be  the  decimal  significance  space  with  mini- 
mum 9 such  that  the  necessary  and  sufficient  condition  in  part  1 of  Theorem  2 

I 

is  satisfied.  Finally,  let  S^,  denote  the  significance  space  associated  with  the 
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reference  software.  We  assume  /3'  and  (3  are  positive  integral  powers  of  two,  and 
we  assume  p'  is  such  that  5^  C 5^/  • 

Then  the  In-and-Out  Conversion  Theorem  allows  us  to  choose  whether  to 
represent  an  argument  set  in  binary  or  decimal.  If  we  choose  binary,  the  base 
conversion  mapping  from  5^  to  5^,  is  trivial.  It  must  be  noted,  however,  that 
this  choice  leads  to  programming  complications  that  are  not  entirely  trivial. 
Also,  conversion  to  decimal  is  necessary  for  human  interpretation.  Therefore  it 
seems  that  the  decimal  choice  should  be  considered.  Assuming  that  rounding 
conversion  mappings  are  correctly  implemented  in  the  computing  environment  of 
the  test  requester  and  also  in  the  computing  environment  used  by  the  software 
test  service,  it  is  immaterial  (except  possibly  for  practical  concerns  involving 
execution  speed)  whether  argument  sets  are  represented  in  binary  or  decimal. 
The  same  remark  is  true  concerning  other  test  data,  such  as  computed  function 
values,  that  are  passed  through  the  communication  interface  from  to  the 
reference  software  or  vice  versa. 

To  summarize,  a test  requester  will  want  to  consider  argument  sets  as  orig- 
inating in  one  of  two  ways: 

Decimal  Origination  The  function  y — f(x)  is  to  be  tested  to  obtain  a gen- 
eral impression  of  its  accuracy  over  parts  of  its  domain.  Here  knowledge 
of  exact  binary  representations  is  not  important,  so  it  is  natural  to  specify 
the  argument  set  in  decimal. 

Binary  Origination  The  function  y = f{x)  is  to  be  tested  at  a set  of  ex- 
actly machine-representable  arguments.  For  example,  if  / is  used  in  an 
application  program,  it  might  be  useful  to  have  the  capability  of  testing 
/ at  the  exact  arguments  that  arise  in  a particular  program  execution. 
Here  decimal  representation  is  still  permissible,  provided  the  conditions  of 
Theorem  2 are  met. 

For  decimal  origination,  use  of  a test  generator  avoids  the  need  to  supply 
arguments  explicitly.  Let  s : [0, 1]”^  — ♦ be  a monotonic  function,  where 

monotonicity  is  defined  componentwise.  To  generate  J test  arguments,  the 
formula 

= s{j/{J  + 1)),  j = l,2,...,J  (12) 

is  used.  Examples  of  univariate  test  generators  are  the  equidistant  generator 
s{t)  = xo(l  — 't)  + xjt  and  the  logarithmic  generator  s{t)  = Xq~^Xj.  These 
generators  produce  J test  arguments  in  the  x-interval  [xo,  xj]  with  equidistant  or 
logarithmic  spacing.  An  element  of  randomness  is  introduced  by  using  a pseudo- 
random number  generator  to  produce  a t-sequence  ti  < t2  < . . . < tj  instead 
of  the  t-sequence  defined  by  tj  = j/[J  -|-  1).  With  sufficiently  careful  coding  of 
the  software  test  service,  and  cissuming  all  rounding  base  conversion  mappings 
are  implemented  correctly,  test  generators  will  produce  identical  argument  sets 
when  executed  either  by  the  test  requester  or  the  software  test  service. 
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We  conclude  this  section  by  introducing  a general  approach  that  can  be  used 
to  specify  argument  sets  on  a proper  submanifold  of  the  function  domain.  Let 

Df  ^ d[^'>  X d\/'>  X ...X  D[1'^  Cn^.  (13) 

denote  the  domain  of  a function  /.  If  the  dimension  m exceeds  1,  it  may 
be  desirable  to  hold  one  or  more  variables  fixed  for  the  duration  of  the  test. 
The  chief  reason  for  such  a procedure  is  that  some  of  the  variables  may  be 
fixed  in  the  application  that  gave  rise  to  the  test.  A straightforward  approach 
would  be  to  list  the  argument  set  with  the  components  corresponding  to  fixed 
variables  remaining  constant  throughout.  However,  another  approach  avoids 
this  unnecessary  specification  of  fixed  variables.  It  has  the  advantage  that  it 
can  be  generalized  to  permit  testing  on  a A:-dimensional  submanifold  in  the 
TO-dimensional  domain,  where  k < m,  and  it  also  can  be  used  to  change  the 
coordinate  system. 

First  we  reorder  the  variables  so  that  the  first  k of  them  in  the  new  order  are 
the  ones  that  will  vary;  the  remaining  m—k  are  held  constant.  We  suppose  that 
k is  given  such  that  1 < k < m.  Let  p be  a permutation,  or  rearrangement,  of 


(1,  2, . . .,  m),  and  let  p p Denoting 

we  have 

the  reordered  variables  ^1,^2,  ■ • 

• J ) 

(r  = l,2,...,m). 

(14) 

and  the  test  is  applied  to  the  function 

y = 9{0  = f{^),  ^ e7^^ 

, xeTZ^,  yen-, 

(15) 

compare  Eq.  (1).  The  test  requester  provides  the  integer  k,  the  permutation 
p,  the  fixed  arguments  Ck+i,^k+2:  • ■ - and  the  argument  set  in  the  domain 
Dg  = X X ...xD[^\ 

Let  us  consider  as  an  example  the  incomplete  gamma  function 

Z 

'y(a,z)  = J e~^t°’~^  dt  (3?a  > 0).  (16) 

0 

Define  a = xi  + ix^,  z = xs  + ix^.  Then 

f{xi,X2,  X3,  24)  = 7(11  + iX2,  X3  -f  iX4).  (17) 

Suppose  a test  is  wanted  in  which  xi  is  held  constant.  Then  k = 3,  the  required 
permutations  are  p = (2,  3, 4, 1),  p = (4, 1,  2,  3),  and  the  function  g is  defined  by 

9(616.^3)  = 7(^4  + + i^a)-  (18) 

Alternatively,  suppose  a test  is  wanted  in  which  the  variables  are  restricted  to 
real  values.  Then  p = p = (1,  3,  2,  4)  and  p(^i,42)  = 7(6  + ^3,^2  + ^44)- 
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6 Concluding  Remarks 

The  proposed  software  test  service  is  undergoing  active  development  at  the 
National  Institute  of  Standards  and  Technology.  The  initial  emphasis  is  on  the 
construction  of  the  communication  interface  and  associated  Web  documents. 
This  substantial  programming  task  is  being  accomplished  with  the  assistance  of 
M.  A.  McClain  of  the  Applied  and  Computational  Mathematics  Division. 

The  envisioned  communication  interface  will  be  accessed  as  a Web  ‘home 
page’  for  the  test  service.  It  will  present  a menu  of  functions  from  which  the 
test  requester  will  choose  by  clicking  the  mouse.  Initially  at  least,  the  menu 
will  follow  the  classification  that  is  used  in  [1].  Special  functions  are  subject  to 
alternative  definitions  arising  from  varying  normalization  criteria,  modification 
by  scaling  functions,  and  other  practical  or  theoretical  considerations.  This  has 
posed  an  identification  problem  in  existing  software  for  evaluating  and  testing 
special  functions  because  of  the  severely  restricted  character  set  used  in  comput- 
ing. An  important  feature  of  Web  documents  is  that  they  support  the  full  range 
of  mathematical  notation.  This  feature  is  being  used  to  avoid  any  ambiguity  in 
the  identification  of  functions  in  our  software  test  service.  It  also  facilitates  the 
possibility  of  offering  a wide  range  of  alternative  function  definitions  for  testing. 

The  software  test  service  will  be  able  to  supply  numerical  function  values 
on  demand  as  well  as  to  evaluate  software.  Thus  the  comparison  software  will 
not  be  relevant  to  all  usage  of  the  service.  The  reference  software  is  essential 
to  all  usage,  and  it  is  very  demanding  to  provide.  It  will  require  a long-term 
research  and  development  effort.  However,  symbolic  computing  environments 
exist  that  support  numerical  computing  to  arbitrary  precision.  Some  have  ex- 
tensive support  for  special  functions,  including  computing  numerical  values  to 
high  precision.  Initially  at  least,  these  environments  will  be  used  to  supply  ref- 
erence values.  Although  they  do  not  meet  our  requirements  for  the  provision  of 
strict  error  bounds,  they  probably  represent  the  best  currently  available  source 
of  reference  software. 

Finally,  in  view  of  the  new  approach  to  testing  that  is  introduced  in  this 
paper,  readers  may  have  opinions,  recommendations  or  criticisms  that  would  be 
useful  in  improving  the  proposed  software  test  service.  The  author  would  be 
most  grateful  for  receiving  all  such  comments. 
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